Media and Pedagogy in Undergraduate Distance Education: A Theory-Based Meta-Analysis of Empirical Literature

This meta-analysis employs a theoretical framework in quantitatively synthesizing empirical studies that investigate the effects of distance education (DE) versus classroom instruction on undergraduate student achievement. Analyses of 218 findings from 103 studies were conducted according to how media were used to support DE pedagogy. The results indicate that the effect sizes for synchronous instructor-directed DE were consistent and not significantly different from zero; in asynchronous DE, media only supporting independent learning was generally less effective than media supporting collaborative discussion among students, although both subsets were significantly heterogeneous. Follow-up analysis of asynchronous DE findings was framed in terms of three patterns of interaction — student-content, student-instructor and student-student. Introduction A longstanding debate in the educational technology literature is whether media or pedagogy makes technology-mediated learning more or less effective. Clark (1983, 1994) argues that pedagogical methods and the application of instructional design principles, rather than the medium used, are what impacts student learning. Others (e.g., Kozma, 1994; Ullmer, 1994; Cobb, 1997) argue that different media attributes make some types of learning easier with one medium than another. For example, television can present information in a dynamic and stimulating manner and therefore may be more appropriate for teaching concepts and skills such as learning a foreign language, where students may benefit most from vicarious experience. The media debate continues in the context of distance education (DE) research, where technology is required for content delivery and communication between students and instructors and among students. While Clark (2000, 2003) continues to argue that media do not influence learning, others (e.g., Smith & Dillon, 1999) claim that the media attributes do. In an attempt to quantitatively synthesize empirical DE studies and identify moderating factors, we conducted a comprehensive meta-analysis of DE comparative studies published between 1985 and 2002 (Bernard et al. 2004). In total, 232 studies containing 688 independent achievement, attitude, and retention outcomes were analyzed. We coded and analyzed 51 study features in an attempt to identify moderating factors of DE effects. To estimate the relative importance of media, pedagogy, and methodological research quality, related study features were classified into the three respective blocks: 8 media features (e.g., use of two-way Undergraduate DE 2 videoconferencing, use of the Web, and use of email), 10 pedagogy and course design features (e.g., systematic instructional design, problem-based learning, and instructor and student activities), and 13 methodology features (e.g., instructor equivalence, student equivalence, and class size equivalence). The three blocks of study features were entered in different orders and analyzed using weighted multiple regression (WMR). The results of the WMRs revealed some interesting insights into the relative importance of these three sets of predictors. Methodological quality explained a significant amount of variance in both synchronous and asynchronous DE outcomes (49% and 12%, respectively, for achievement) when the block of 13 methodology study features was entered at step 1. Pedagogy explained a significant 10%-13% of variance whether entered at step 1 or after controlling for methodology quality. Media were only significant when entered at step 1. These results support Clark’s (2000) view that research methodological quality often confounded studies on technology effects and that pedagogy features were more important than media in predicting student achievement. However, further examination of the individual predictors also showed that after controlling for methodology quality, a few media-related features significantly predicted DE student achievement. What is more puzzling is that while some media (e.g., use of one-way TV or video) predicted DE student achievement positively, some (e.g., use of telephone to contact the instructor) predicted DE student achievement negatively. These results suggest more intricate and complex relationships between media and pedagogy, calling for further analyses for better and more holistic understanding. The purpose of this study is to extend the work in Bernard et al. (2004) to further explore the relationship between media and pedagogy in influencing DE student achievement using a theoretical framework. In particular, we attempt to identify how media were used to support different types of DE pedagogy, the relative effects of different types of media-supported DE pedagogy, how different types of interactions (Moore, 1989) were supported via media and pedagogy, and the relative impact of media and pedagogy on DE student achievement. In the following sections, we discuss the rationale for focusing on types of media-supported DE pedagogy and types of interaction as a framework in coding and analyzing the data in this study. Types of DE Delivery Media and Pedagogy DE delivery media have changed dramatically as technology has advanced. Nipper (1989) categorized them into three generations. First generation DE refers to the early days of print-based correspondence study. Characterized by the establishment of the Open University in 1969, second generation DE refers to the period when print materials were integrated with broadcast TV and radio, audio and video-cassettes, and increased student support. Third generation DE was heralded by the invention of Hypertext and the rise in the use of teleconferencing (i.e., audio and video). To this, Taylor (2001) adds the fourth generation, characterized by flexible learning (e.g., computer-mediated communication [CMC], Internet accessible courses) and the fifth generation (e.g., interactive multimedia online, Internet-based access to online resources). Generations three, four and five represent the potential for DE to move away from authoritarian and non-interactive courses to those involving a high degree of student control and two-way communication, as well as group-oriented processes and greater flexibility in learning. Another way of categorizing DE is according to what pedagogy medium is used and its purpose. Is it used to support instructor-directed instruction, individualized learning, or Undergraduate DE 3 collaborative discussion among the students? Traditionally, DE has been designed to support individualized self-directed learning (Keegan, 1996). This is true for the early correspondence model and the multimedia model that uses audio and videotapes. It is still true for some of the DE applications that use only Web-based resources. The individualized learning model affords the highest degree of flexibility for anytime, anywhere, and anyplace learning. However, because the individualized model is also low in interaction, successful completion requires a high degree of student autonomy and self-regulation (Moore, 1989). This model is mostly used in adult education, especially with students who are working full time, or otherwise would not have access to regular university instruction. Instructor-directed two-way interactive satellite television and videoconferencing provide the closest resemblance to traditional classroom instruction (Simonson, Schlosser, & Hanson, 1999). In videoconferencing, the instructor usually teaches a class of students at the host site and one or more classes of students at the remote site or sites simultaneously. The advantage of this type of DE is that the instructor is able to observe immediately how students react to the instruction and to adjust if necessary, just as instructors regularly do in the classroom. This paradigm emerged on U.S. university campuses (e.g., Penn State) in the 1950s (i.e., closedcircuit television) and continues as a common pattern in higher education today. The collaborative learning paradigm in DE is more recent. It is a result of both advances in CMC technology and a shift in education towards a more collaborative learning model. According to recent constructivist and cultural learning theories, learning occurs through interacting with others (Resnick, Levine, & Teasley, 1993). Greater effectiveness is achieved when students collaborate and learn from each other through discussions that challenge ideas and create multiple perspectives. The collaborative paradigm may use only CMC or CMC in combination with other delivery media such as Web-based resources or email. The above categorization takes into account media attributes and the pedagogy they are used to support. This scheme may help explain the different effects across DE studies employing various media. It can also deal with the use of multiple media and overcomes the limitation of treating each medium as if it was used in exactly the same way. Depending on its use, broadcast television could facilitate asynchronous individualized instruction with repeatable tapes or streaming video, or synchronously in instructor-directed instruction with two-way audio. Similarly, Web-based DE could use Web-based resources alone for individualized learning or with CMC to facilitate collaborative discussion among students. Types of Interaction in DE Interaction is the defining component of all forms of education. Moore (1989) describes three types of interaction in DE that play important roles in student learning: student-content interaction, student-instructor interaction, and student-student interaction. Student-content interaction refers to students interacting with the subject matter under study to construct meaning, relating it to personal knowledge, and applying it to solve problems. Student-content interaction may include reading informational texts, using study guides, watching videos, interacting with computer-based multimedia, and completing assignments and projects. Student-instructor interaction traditionally focused on classroom-based dialogue between students and the instructor. In DE environments, student-instructor interaction may be synchronous through the telephone, videoconferencing, and chats, or asynchronous through correspondence, email, and discussion boards. Face-to-face interaction between students and instructors is also possible in some DE environments. According to Moore (1989; Moore & Undergraduate DE 4 Kearsley, 1996, 2005) and several other DE theorists (e.g., Anderson, 2003; Holmberg, 1989, 2003), student-instructor interaction facilitates student learning not only by providing cognitive guidance and feedback, but also motivational and emotional support. Student-student interaction refers to interaction among individual students or among students working in small groups. In correspondence courses, this interaction is often absent. In later generations of DE, including two-way videoconferencing and Web-based courses, studentstudent interaction can be synchronous, as in videoconferencing and chatting, or asynchronous through discussion boards or email messaging. With DE becoming popular in mainstream education with on-campus students, student-student interaction may also include face-to-face contact. According to social theories of learning and distributed cognition (Salomon, 1993), student-student interaction is often desirable both for cognitive purposes and for motivational